1. Field of the Invention
The present invention relates to an image processing system, an image processing method, and a medium having an image processing control program recorded thereon. Particularly, the invention is concerned with an image processing system and method for executing an optimum image processing with use of a computer, as well as a medium having an image processing control program recorded thereon.
2. Description of the Prior Art
Heretofore, for image data of natural pictures such as photographs and image data of drawing type unnatural pictures, there have been known software such as photoretouch software for performing various effect processing. In such softwares, image data are displayed on a display or the like, to which an operator applies a desired processing to form nicexe2x80x94looking image data.
For example, in the case of reading a photograph through a scanner to obtain electronic image data, it is sometimes desired to highlight the contrast and vividness of the image.
As a method for enlarging the
contrast of image data there is known the method disclosed in Japanese Patent Publication No.66318/95, in which the relation the luminance y before transformation and the luminance y after transformation are correlated with each other according to the following expression (1), and the luminance of image data is transformed on the basis of parameter xe2x80x9caxe2x80x9d or xe2x80x9cbxe2x80x9d selected by an operator, whereby there is obtained a contrast-highlighted image from image data weak in contrast.
yxe2x80x2=ay+bxe2x80x83xe2x80x83(1)
On the other hand, as a method for highlighting saturation to highlight the vividness there is known a method wherein the value of a desired color component is increased in the case where the color components of image data are represented in terms of gradation data of red (R), green (G) and blue (B).
More particularly, when the gradation data is in the range of 0 to 255, 20 is always added to the gradation data of red to make red color more vivid, or 20 is always added to the gradation data of blue to make blue color more vivid.
Thus, among various effect processing there is included one which transforms natural pictures into more beautiful pictures, while some ages are not required to be retouched, such as drawing type images.
In the latter case, in an image processing system using the above conventional software, the operator is required to judge the type of image on the display and select an appropriate effect processing from among various ones or judge the degree of the processing. Thus, it has been impossible to automate this operation.
For example, it has so far been impossible to judge automatically whether contrast should be highlighted or not or whether vividness should be highlighted or not.
In highlighting contrast, the conventional methods merely adopt the technique of providing plural settings different in strength of contrast beforehand and then switching over from one to another. Thus, it has been impossible to automatically select the most preferred one corresponding to the actual image data. Particularly, when luminance is changed in accordance with the foregoing expression (1), the result is such that only brightness is highlighted in the case of an image which is bright as a whole, or only darkness is highlighted in the case of an image which is dark as a whole.
Needless to say, the strength of contrast can be adjusted in television for example, but the parameter xe2x80x9cbxe2x80x9d, namely offset, is uniform and it has been impossible to effect an optimum highlighting for each individual image.
Also as to highlighting vividness, a man is required to judge to what degree of highlighting is to be performed with respect to each image data, and it has been impossible to select the best suited highlighting automatically.
Further, as to not only the contrast enlarging processing but also the saturation highlighting processing, there is a tendency to increase in value of image data. Therefore, if saturation is to be highlighted after highlighting of contrast, a loud image results. This is not desirable.
In this connection, in the above conventional image processing system, if one is adjusted to an appropriate condition and thereafter the other is adjusted, the result of the previous adjustment goes wrong and thus it is difficult to make adjustment as desired.
In U.S. Pat. No. 5,181,105 there is disclosed a technique of diminishing the contrast width of image data automatically in view of the point that a color space reproducible in a CRT display is larger than that in a video printer. In this case, all that is required is merely causing image data to enter the color space in the video printer which is known in advance. Thus, automation is feasible because only such reduction of the contrast width is needed irrespective of the type of image. Such an automation has nothing to do with the foregoing effect processing that intends to make the most of the image quality of the image data. Thus, the application of the aforesaid technique is meaningless.
Accordingly, it is an object of the present invention to provide an image processing system, an image processing method, and a medium having an image processing control program recorded thereon, capable of judging the type of image automatically on the basis of image data and performing an optimum image processing.
The image processing system according to the present invention comprises a number-of-color detecting means which inputs image data representing information of each of pixels resolved in a dot matrix form from an image and which detects the number of colors used while regarding information corresponding to the luminance of each pixel as color, and an image discriminating means for judging the type of image on the basis of the number of colors thus detected.
The image processing method according to the present invention performs a predetermined image processing for image data which represents information of each of the pixels resolved in a dot matrix form from an image. The same method comprises inputting the said image data, detecting the number of colors used while regarding information corresponding to the luminance of each pixel as color, and judging the type of image on the basis of the number of colors thus detected.
Further, the medium according to the present invention has an image processing control program recorded thereon for inputting to a computer image data which represents information of each of the pixels (resolved in a dot matrix form from an image and for performing a predetermined processing. The said image processing control program includes the step of inputting the said image data and detecting the number of colors used while regarding information corresponding to the luminance of each pixel as color and the step of judging the type of image on the basis of the number of colors thus detected.
In the present invention thus constructed, when the number-of-color detecting means detects image data which represents information of each pixel resolved in a dot matrix form from an image, it detects the number of colors used while regarding information corresponding to the luminance of each pixel as color, and the image discriminating means judges the type of image on the basis of the number of colors thus detected.
The number of colors used differs depending on the type of image. For example, in the case of a natural picture such as a photograph, even if the object to be photographed is of the same blue color, plural colors are detected due to a shadow and thus a considerable number of colors may be used. On the other hand, in the case of a drawing type data or a business graph, a limit is encountered in the number of colors used because it has been drawn with colors designated by an operator.
According to the present invention, from the standpoint of the number of colors used, it is possible to judge the type of image automatically on the basis of image data. Needless to say, images including a large number of colors used correspond to natural pictures. If the number of colors used is large, the image discriminating means judges that the image concerned is a natural picture, while if the number of color used is small, the image discriminating means judges that the image concerned is a business graph or the like. In connection with this image data, information corresponding to each pixel represents of color directly or indirectly. Image data covers component values of plural elementary colors, coordinate values of a known absolute color space, and the brightness of a monotone.
According to one method for detecting the number of colors used, actual colors of matrix pixels are judged and totaled in a histogram form for detection. However, in the sense of judging the type of image, an exact value is not always needed.
On the other hand, each color and luminance are correlated with each other, and although plural colors correspond to a single luminance value, there does not occur that the number of luminance values is large and that of color is small. Besides, it is actually inconceivable that an image is constructed of only colors of the same luminance value. Thus, it can be said that a rough tendency as to whether the number of colors used is large or not can be judged in terms of luminance values.
According to the present invention, therefore, by replacing the number of colors used with luminance, it becomes easy to effect totaling even when the number of reproducible colors is very large. Thus, the xe2x80x9cnumber of colors usedxe2x80x9d as referred to herein is of a broad concept corresponding to the number of colors.
In totaling luminance values, image data sometimes include luminance data directly or may include luminance data only indirectly. In the case of direct luminance data, it suffices to make transformation thereof, or even in the case of indirect luminance data, it suffices to first transform it into luminance data and thereafter performs a predetermined luminance transformation. However, it is not that the transformation of luminance must be extremely accurate, but it can be said that a rough transformation will do.
Therefore, it is another object of the present invention to provide an image processing system capable of obtaining luminance more easily even when image data does not include any element that represents luminance directly.
In the image processing system according to the present invention, in the case where image data is represented in terms of plural component values corresponding to luminance, the number-of-color detecting means determines a luminance by weighted integration of the said component values.
Despite the merit of using luminance, if there is included no luminance component in the parameter of image data, it is necessary to make color transformation. Usually, a large number of processings are required for color transformation. In this connection, since a slight error is allowed at the time of utilizing luminance, in the case where image data is represented in terms of plural component values corresponding to luminance, luminance can be determined by weighted integration of the component values without requiring a large number of processings. For example, where image data is represented by the gradation data of RGB (red, green, blue), it can be said that the component values of red, green and blue each correspond to luminance. For this reason it can be said that a linear addition of the component values represent luminance enough. Thus, this can be an extremely easy method for transformation.
There also are available other methods for simplifying the detection of the number of colors used. For example, a lattice shape sporadic in each axial direction is considered in a three-dimensional color space, and it is assumed that each group represents one color. By so doing, it is no longer required to total a very large number of colors such as, for example, 256 gray scales in each axis.
On the other hand, in the case of a large image data; even a mere totaling of the number of colors used requires a considerable volume of processing. In view of this point it is a further object of the present invention to provide an image processing system capable of totaling the number of colors used in a more simple manner.
According to this image processing system, the number-of-color detecting means makes sampling of pixels substantially uniformly from among all the pixels and the number of colors is detected on the basis of the image data concerning the sampled pixels.
Since the number of colors used is not always required to be strict, it is not always necessary that the totaling of colors throughout the whole image data be conducted for all the pixels.
For ensuring a certain degree of randomness by such sampling it is necessary to ensure an appropriate sampling ratio. Since an image is planar, the associated image data are inevitably distributed in both longitudinal and transverse directions. But in determining a sampling ratio, by ensuring an appropriate number of pixels sampled at least on the shorter side, it becomes possible to maintain a sureness proportional to the sampling ratio to be determined.
On the other hand, the image data having been subjected to image processing can be applied not only to display but also to various other purposes, including printing. In printing, it is necessary to make transformation into a color space of printing ink different from that of the original image data. But in relation to the number of colors used, the type of image exerts a great influence on the amount of processing in such color transformation. In view of this point it is a further object of the present invention to provide an image processing system capable of selecting and executing an optimum color transformation processing according to the type of image.
This image processing system comprises a pre-gray level transforming means provided with a table having lattice points in a color-specification space of data to be transformed in association with colorimetric gradation data in a color-specification space of data after transformation, the pre-gray level transforming means performing a gray level transformation of colorimetric gradation data before transformation into those corresponding to the lattice points in the said table, then referring to the same table and reading the corresponding colorimetic gradation data for color transformation; an interpolating color transformation means capable of making color transformation into corresponding colorimetic gradation data by interpolating calculation between lattice points in the above table and having a storage area capable of reading data at high speed and storing information on that color transformation, the interpolating color transformation means utilizing a cache for color transformation by an interpolating calculation in the case of information not stored in the said storage area; a color transformation selection control means which, in the case of a natural picture having a large number of colors used in the image data, makes color transformation with use of the pre-gray level transforming means and which, in the case of an image having a small of colors in the image data and being not a natural picture, makes color transformation with use of the interpolating color transformation means.
In the case of making color transformation between different color-specification spaces, a table is provided in which gradation data in the color-specification space after transformation are associated with the lattice points in the like space before transformation, and corresponding data at predetermined lattice points are read out successively by referring to the said table, whereby it is made possible to effect transformation. However, if there are provided lattice points corresponding to all the gray scale values of the gradation data before transformation, the table becomes too large. For this reason, for example by error diffusion, the pre-gray level transforming means makes a gray level transformation of the gradation data before transformation into gradation data corresponding to the lattice points in the table to thereby reduce the size of the table. Also as to the other gradation data than at the table lattice points, transformation can be done by interpolating calculation. In the case where the number of colors is large, the gray level transformation executed by error diffusion for example is characteristic in that the amount of calculation is sure to become smaller.
However, this method requires that the error diffusion processing must be performed for all the pixels. Therefore, in the case of an unnatural picture which is small in the number of colors used, the result obtained by a single transformation can be utilized many times. In such a case, the interpolating color transformation means utilizing a cache may be advantageous at a point of processing speed, from the function of repeating the processing of storing the result of color transformation in the high-speed readable storage area while performing an interpolating calculation and reading it from the storage area if necessary.
According to the present invention, therefore, in the case of a natural picture using a large number of colors, it is possible to make color transformation at a minimum amount of calculation in accordance with the error diffusion technique for example, while in the case where the number of colors used is small, the result of transformation can be utilized repeatedly by using a cache. Thus, the amount of processing can be kept to a minimum by a color transformation method adopted in accordance with the type of image.
Judgment of the type of image can also be utilized for judgment as to whether edge highlighting is to be performed or not. It is therefore a further object of the present invention to provide an image processing system capable of judging whether edge highlighting is to be performed or not.
This image processing system comprises a natural picture discriminating means which, in the case where the number of colors judged on the above image date is a predetermined number of colors or more, judges that the image data represents a natural picture, and an edge emphasizing means which, when the image data is judged to be a natural picture by the natural picture discriminating means, determines a low-frequency component on the basis of a marginal pixel distribution with respect to each of the pixels which constitute the image data, and diminishes the low-frequency component, thereby eventually making a modification so as to enhance the edge degree of each pixel.
In the present invention constructed as above, when the number of colors judged on the image data is a predetermined number or more, the natural picture discriminating means judges that the image data represents a natural picture. When this judgment is made by the natural picture discriminating means, the edge highlighting means determines a low-frequency component on the basis of a marginal pixel distribution with respect to each of the pixels which constitute the image data, and diminishes the low-frequency component, thereby eventually making a modification so as to enhance the edge degree of each pixel. Thus, when the image data represents a natural picture, there is performed an image processing for edge highlighting, while when it is an unnatural image, there is not performed edge highlighting.
In the case of a photograph as an example, certain photographing conditions afford a blurry image whose edge is blunt. To prevent this inconvenience, an image processing is sometimes conducted. More specifically, when the number of colors in image data detected by the number-of-color detecting means is large, it is judged that the image data represents a natural picture, and in this case there is performed an image processing for edge highlighting. Needless to say, the degree of edge highlighting can be changed as necessary.
It is not necessary that the automation based on image data be limited to the judgment of image type. For example, in the case of a natural picture, it is preferable that its contrast be adjustable automatically. It is therefore a further object of the present invention to provide an image processing system, an image processing method, and a medium having an image processing control program recorded thereon, capable of adjusting contrast automatically on the basis of image data.
This image processing system comprises a luminance distribution totaling means which inputs image data representing information of each of pixels resolved in a matrix form from an image and which totals luminance values of the pixels into a luminance distribution as a whole on the basis of the luminance of each pixel, and an image data luminance transforming means which, when the thus-totaled luminance distribution is not widely dispersed in an effective luminance range of the image data, transforms the luminance information of each pixel in the image data so that the above luminance distribution covers widely the luminance range.
The image processing method according to the present invention performs a predetermined image processing for image data which represents information of each of pixels resolved in a dot matrix form from an image. According to this method, luminance values of the pixels are totaled into a luminance distribution as a whole on the basis of the luminance of each pixel, and when the thus-totaled luminance distribution is not widely dispersed in an effective luminance range of the image data, the luminance information of each pixel in the image data is transformed so that the luminance distribution widely covers the said luminance range.
The medium according to the present invention has an image processing control program recorded thereon for inputting to a computer image data which represents information of each of pixels_resolved in a dot matrix form from an image and for performing a predetermined image processing. The image processing control program comprises the steps of inputting the image data and totaling luminance values of the pixels into a luminance distribution as a whole, and when the thus-totaled luminance distribution is not widely dispersed in an effective luminance range of the image data, transforming the luminance information of each pixel in the image data so that the luminance distribution widely covers the said luminance range.
In the present invention thus constructed, what is called the width of contrast in the image data can be digitized to some extent by determining a luminance distribution in the image data, and after the digitization can be done, the said distribution is enlarged correspondingly to a reproducible range. It is not that the digitization always requires concrete numerical values. In the course of processing, the data may be treated as a numerical value or as the magnitude of signal.
To be more specific, for determining a related luminance distribution in handling image data of a predetermined image, the luminance distribution detecting means detects a luminance distribution of the image data in the unit of pixel. Then, using this detected luminance distribution, the image data luminance transforming means judges an amount of enlargement of the luminance distribution in an effective available range and makes transformation of the image data.
Thus, by obtaining a luminance distribution of the image data in the unit of pixel, it is made possible to judge the so-called contrast width ranging from the highest luminance to the lowest luminance, and an enlargement ratio can be determined by comparison with the effective luminance range. Therefore, after that, all that is required is merely enlarging the luminance distribution so as to afford the enlargement ratio.
For enlarging the luminance distribution there may be adopted several methods insofar as the luminance distribution of image data is enlarged in an allowable range. In view of this point it is a further object of the present invention to provide a luminance distribution enlarging method and an example thereof.
In the image processing system according to the present invention, the image data luminance transforming means compares a statistical width of the luminance distribution detected with the width of the aforesaid luminance range and determines an enlargeable degree as the enlargement ratio. At the same time, the image data luminance transforming means determines an adjustment value to make adjustment so that the upper and lower ends of the thus-enlarged luminance distribution are within the luminance range in question, and modifies the luminance of each pixel. As a n example, given that the range of a reproducible luminance is y range, the luminance Y after transformation is obtained from th luminance y before transformation and a maximum value y max and a minimum value y min in the luminance distribution range in accordance with the following expression:
Y=ay+bxe2x80x83xe2x80x83(2)
where,
a=y range/(ymaxxe2x88x92ymin)xe2x80x83xe2x80x83(3)
b=axc2x7ymin or yrangexe2x88x92axc2x7ymaxxe2x80x83xe2x80x83(4)
In the above transformation expressions, Y=0 if Y less than 0 and Y=yrange if Y greater than yrange.
According to the concept of the transforming method, both enlargement ratio and adjustment value are determined and the luminance of each pixel is modified on the basis of those values. An example is a linear enlargement. Although the transformation expression itself is the same as the conventional one, a significance resides in that the parameter thereof is selected by the image data luminance transforming means. Irrespective of the: selection of b, Y=0 if y=ymin and Y=yrange if y=ymax. And the luminance distribution expands uniformly within the y range which is a reproducible luminance range. The transformation in this example is a linear transformation in a narrow sense and no limitation is placed thereon. It is also possible to make a non-linear transformation in a broad sense. The transforming expression in question is merely one example and it goes without saying that even other transforming expressions of the same meaning may also be employed.
According to the present invention thus constructed it is possible to enlarge the luminance distribution effectively within a predetermined range of gradation.
However, a mere expansion of contrast may sometimes be unable to cope with the case where the whole is bright or dark. It is therefore a further object of the present invention to adjust the whole brightness automatically.
In the image processing system according to the present invention, the image data luminance transforming means determines a maximum distribution luminance of the luminance y before transformation and executes xcex3 correction to change the luminance distribution so that the range to which the maximum distribution luminance gives a desired brightness, thereby obtaining the luminance Y after transformation.
For judging whether image data is bright or not as a whole, the maximum distribution luminance of luminance y before transformation is utilized, and if the maximum distribution luminance is on the bright side, there is made xcx9cy correction to render the whole rather dark, while if the maximum distribution luminance is on the dark side, there is made y correction to render the whole rather bright. In this way the entire brightness is corrected automatically, which cannot be attained by only the highlighting of contrast. It is optional whether the maximum distribution luminance of luminance y before transformation is to be obtained in terms of median or in terms of a mean value.
Thus, according to the present invention, the degree of brightness which cannot be adjusted by only the highlighting of contrast is also adjustable.
In transforming the luminance by any of various methods, it is also possible to first calculate and store the luminance Y after transformation within the range of the luminance y before transformation and then call up the correlation at the time of transformation. It is not impossible to calculate the luminance every time in accordance with transforming expressions, but the range of values which the luminance distribution can take has already been determined. Therefore, if the luminance Y after transformation is calculated and stored beforehand on the basis of the luminance y before transformation, it becomes possible to effect transformation by merely calling up the correlation when the transformation is to be made.
When the luminance distribution of image data is reviewed statistically, it is proper to consider that the luminance distribution extends up to both-end positions in a reproducible luminance range although such a case is seldom encountered. It follows that both ends of an actual luminance distribution always correspond to both ends in a reproducible luminance range. If both ends in question are adopted, the enlargement ratio substantially becomes equal to xe2x80x9c1xe2x80x9d and it is impossible to attain the expected effect. It is therefore a further object of the present invention to obtain a substantial luminance distribution range which is enlargeable.
In the image processing system according to the present invention, the luminance distribution totaling means regards positions inside by a predetermined distribution ratio from the actual ends of a luminance distribution as end portions at the time of determining the above luminance distribution.
In the present invention thus constructed, by removing the predetermined distribution ratio at both ends, the skirt portion of an extremely reduced distribution is ignored moderately from the statistical point of view. Therefore, this range is used as a criterion in judging the degree of enlargement.
The predetermined distribution ratio is not specially limited if only it permits the skirt portion of an extremely reduced distribution to be ignored. For example, it may be the number of pixels corresponding to a certain ratio of the total number of pixels, or the positions where the distribution ratio is below a certain ratio may be regarded as end portions.
Thus, according to the present invention it is possible to obtain a luminance distribution more effective for judgment.
On the other hand, in an actual image there are both highlight portion and high-shadow portion, and the human eyes are easy to sense a delicate difference in these portions. Therefore, if an intentional enlargement reaching the end portions in a reproducible luminance range is performed, the highlight portion gives rise to a bald white impression, while the high-shadow portion affords a solid black impression.
It is therefore a further object of the present invention to provide an image processing system capable of improving the expressibility of such highlight and high-shadow portions.
In this image processing system, a luminance distribution range to be enlarged is set inside the end portions of the actual reproducible range by a predetermined amount. By so doing, an intentional enlargement is no longer performed at both end portions.
Thus, according to the present invention it is possible to prevent collapse of the highlight and high-shadow portions.
In some case a narrow contrast is inevitable. For example, in an evening scene it is natural that the luminance distribution width should be narrow. If it is enlarged to a greater extent than necessary, a daytime scene will result. It is therefore a further object of the present invention to provide an image processing system capable of preventing the impression of an image from being transformed to an unnatural degree.
In this image processing system, a limit is imposed on the enlarging degree of luminance distribution. By so doing, image data whose narrow contrast is inevitable is prevented from being enlarged its luminance distribution to an excessive degree. Needless to say, the above phenomenon is not limited to evening scenes but can be observed also in other cases. The occurrence of such a phenomenon is prevented by placing a limitation on the enlarging degree of luminance distribution.
Thus, according to the present invention it is possible to prevent excessive highlighting of contrast which would causes a change in atmosphere of the image concerned.
If a luminance distribution is used to the maximum extent within its reproducible range, there no longer remains a margin for enlargement of the luminance distribution. However, if a limit is imposed on the degree of enlargement, there remains a margin which permits enlargement of the luminance distribution. In other words, there remains freedom for selecting actual range to be used for the enlargement, and this selection may cause a change in atmosphere of image. It is therefore a further object of the present invention to provide an image processing system capable of automatically adjusting the margin which permits enlargement of a luminance distribution even when a limitation is placed on the degree of enlargement.
In this image processing system, a corresponding position of a luminance distribution range before transformation and that after transformation in a reproducible range are maintained to prevent the center of the luminance distribution from being changed.
The luminance distribution center of image in such a sense can be grasped in various ways. For example, the luminance distribution may be enlarged so that an enlargeable range ratio remaining at each of upper and lower ends of the luminance distribution range before transformation is retained also after transformation.
To be more specific, since it suffices if only the center is retained, it can be said unnecessary to directly catch and hold the center. Conversely, an enlargeable range remaining at each of upper and lower ends of the luminance distribution range before transformation is noted and the luminance distribution is enlarged so as to retain the ratio of that range also after transformation, thereby holding the center substantially.
As another example, it is preferable that the automatic judgment based on image data be applied to judging the degree of highlighting at the time of highlighting the vividness of a natural picture. It is therefore a further object of the present invention to provide an image processing system, an image processing method and a medium having an image processing control program recorded thereon, capable of adjusting the vividness automatically on the basis of image data.
This image processing system comprises a saturation distribution totaling means which inputs image data representing information of each of pixels resolved in a dot matrix form from an image and which then totals saturations of the pixels into a saturation distribution as a whole, a saturation transformation degree judging means for judging a saturation transformation degree of image data from the saturation distribution obtained by the saturation distribution totaling means, and an image data saturation transforming means which modifies saturation information contained in the image data on the basis of the transformation degree thus judged and transforms the modified information into a new image data.
The image processing method according to the present invention is for performing a predetermined image processing for image data which represents information of each of pixels resolved in a dot matrix form from an image. The method comprises totaling saturations of the pixels into a saturation distribution as a whole, then judging a saturation transformation degree for the image data from the saturation distribution thus obtained, then modifying saturation information contained in the image data on the basis of the transformation degree thus judged and transforming it into a new image data.
Further, the medium according to the present invention has an image processing control program recorded thereon for inputting to a computer image data which represents information of each of the pixels resolved in a dot matrix form from an image and for performing a predetermined image processing, the program comprising the steps of inputting the image data and totaling saturations of the pixels into a saturation distribution as a whole, judging a saturation transformation degree for the image data from the saturation, distribution thus obtained, and modifying saturation information contained in the image data on the basis of the transformation degree thus judged and transforming it into a new image data.
In the present invention constructed as above, after the saturation distribution totaling means has totaled saturations of the pixels in the image data into a saturation distribution as a whole, the saturation transformation degree judging means judges a saturation transformation degree for the image data from the saturation distribution thus obtained, and the image data saturation transforming means transforms the image data in accordance with the transformation degree thus judged. That is, for each image, an optimum transformation degree is judged from a saturation distribution of image data and the image data is transformed on the basis thereof.
According to the present invention, therefore, it is possible to provide an image processing system wherein a saturation transformation best suited for each image can be done using a totaled saturation distribution.
In totaling saturations of pixels in image data into a saturation distribution, if the image data has saturation parameters, it suffices to total the parameters. Even in the case where the image data does not possess such parameters, it is possible, for example, to make a color transformation from another color specification space for a color specification space having such a saturation parameter and then total saturation parameters after the transformation. However, a considerable volume of processing is needed for the color transformation. It is therefore a further object of the present invention to total saturations without color transformation.
In the image processing system according to the present invention, the saturation of each pixel is judged in accordance with the saturation of a warm color hue in color components.
Although it is difficult to judge saturation without a change of color specification space, the human visual characteristic tends to regard the difference between a warm color hue and a non-warm color hue as being vivid, so it is relatively convenient to judge saturation on the basis of such difference.
Thus, according to the present invention it is possible to total saturations in accordance with the human visual characteristic.
In the image processing system according to the present invention, saturation (=X) is represented by the following expression, provided the color components of image data are red (R), green (G) and blue (B):
X=|G+Bxe2x88x922xc3x97R|xe2x80x83xe2x80x83(5)
In the color specification space of RGB which is often used in a computer, there occurs a saturation-free state upon coincidence of the components and in the other cases there occurs saturation. In this connection, it is possible to judge the difference from the saturation-free state and thereby determine saturation, but the use of the relationship |G+Bxe2x88x922xc3x97R| is preferred because the coincidence of the components affords a minimum value irrespective of component values, also because a maximum value is obtained in red and light blue monocolor and further because also in the case of blue or green there is presented a relatively large saturation. Needless to say, based on the same way of thinking, a like simplicity is attained by:
Xxe2x80x2=|R+Bxe2x88x922xc3x97G|xe2x80x83xe2x80x83(6)
Xxe2x80x3=|G+Rxe2x88x922xc3x97B|xe2x80x83xe2x80x83(7)
Judging from experimental results, however, the best results are obtained by the expression (5). It can be said that this is supported by the human visual characteristic of recognizing vividness according to the saturation of a warm color hue.
According to the present invention, therefore, in the case of adopting the color specification of RGB, it is possible to decrease the calculation volume and total saturations in a simple manner.
Needless to say, as is the case with enlargement of luminance distribution, it is not always necessary that the saturation distribution in an image be obtained with respect to all the pixels of image data, insofar as it is possible to total saturations of pixels in image data into a saturation distribution. No special limitation is placed on a concrete totaling method.
In judging the degree of saturation transformation from the state of saturation distribution, there may be adopted any of various analyzing methods for analyzing the state of distribution concretely. Now, it is a further object of the present invention to give an example of a concrete analyzing method for the state of distribution.
In the image processing system according to the present invention, the saturation transformation degree judging means strengthens the degree of saturation highlighting in the case of a small saturation at a predetermined ratio from the upper end in the totaled saturation distribution, while when the said saturation is large, it weakens the saturation highlighting degree, thereby judging the degree of saturation transformation.
Thus, in judging the state of saturation distribution, a portion of the distribution is taken out from a large saturation side, then a judgment is made as to whether the saturation of that portion is large or small, and the entire saturation is detected from the result of the judgment, then the saturation is highlighted if it is small, while if the saturation is sufficiently large, it is judged that highlighting is not necessary. Needless to say, there may be adopted any other method, for example another statistical method, to judge the saturation tendency of the entire image data. Further, image data may be grasped partially, not wholly.
Thus, according to the present invention, judging the degree of saturation transformation at a predetermined ratio from the upper end in the totaled saturation distribution facilitates the judging.
There also are various other concrete methods for transforming the saturation of image data. For example, the transformation of saturation may be performed by radial displacement according to the foregoing degree of transformation within the Luv space in the standard calorimetric system.
To be more specific, if image data has a saturation parameter, this parameter may be transformed, but in the Luv space as the standard calorimetric system having a parameter of luminance or lightness and a parameter of hue in a plane coordinate system with respect to each luminance, the radial direction corresponds to saturation. In the Luv space, therefore, the transformation of saturation is performed by radial displacement.
The reason why the Luv space is adopted is that luminance is independent and that the displacement of saturation exerts no influence on luminance. In the case of using the Luv space, however, it is necessary to make transformation if there is no correspondence of the original image data thereto.
On the other hand, in the case where image data is expressed by equal hue components, as is often the case with image data, there may be adopted a method wherein components values exclusive of a non-saturation component are shifted according to the degree of transformation.
In the case where image data is represented by component values of plural, nearly equal hue components like RGB, it can be said that there is a saturation-free component. It follows that the component values exclusive of the saturation-free component exert an influence on saturation. In this case, saturation is transformed by displacing the component values.
As an example of coping with such saturation-free component there may be adopted a method wherein a minimum component value in plural hue components is subtracted from the other component values and the differential value thus obtained is increased or decreased to effect saturation transformation.
Of the plural color components, the minimum component value is also contained in the other color components, and such minimum component values are merely combined together and constitute a saturation- free gray. Thus, the differential value based on the other colors and exceeding the minimum component value exerts an influence on saturation, and by increasing or decreasing the differential value there is performed transformation of saturation.
According to another example, a corresponding value of luminance is subtracted from each component value and the differential value obtained is increased or decreased to perform the transformation of saturation. A mere displacement of component values exclusive of a saturation-free component will cause a change in luminance. For this reason, if a corresponding value of luminance is subtracted beforehand from each component value and the transformation of saturation is performed by increasing or decreasing the resulting differential value, it becomes possible to store luminance.
As to the luminance or lightness and the saturation are in a relation such that the color specification space assumes an inverted cone shape up to a certain range. It can be said that the lower the luminance, the larger the component values of hue. In such a case, if an attempt is made to apply a transformation degree proportional to a small value of saturation, there is a fear of breaking through the conical color specification space. Therefore, when the luminance is low, the transformation degree of saturation may be weakened to prevent the occurrence of such inconvenience.
The transformation degree of saturation is judged by any of the foregoing various methods. However, if saturation is highlighted to a greater extent than necessary even when it is weak, there will not be obtained good results.
As to both saturation highlighting process and contrast enlarging process, there is a tendency to increase the value of image data. Therefore, if contrast is highlighted after highlighting of saturation, a loud image will result. This is not desirable.
However, since both processings are conducted each individually, one is first adjusted to an appropriate state and thereafter the other is adjusted, so it is unavoidable that the result of the previous adjustment will go wrong.
It is therefore a further object of the present invention to provide an image processing system, an image processing method and a medium having an image processing control program, capable of obtaining a desired image more easily.
The image processing system according to the present invention inputs image data which represents information of each of the pixels resolved in a dot matrix form from an image and then executes a predetermined image processing. The image processing system is provided with a contrast enlarging means for enlarging the luminance distribution in the image data, a saturation highlighting means for highlighting saturation in the image data, and a highlighting process suppressing means for suppressing the luminance distribution enlarging operation and the saturation highlighting operation with respect to each other.
The image processing method according to the present invention comprises inputting image data which represents information of each of pixels resolved in a dot matrix form from an image, enlarging the luminance distribution in the image data and highlighting saturation in the image data. The luminance distribution enlarging operation and the saturation highlighting operation are performed in a correlated manner so as to suppress each other.
Further, the medium according to the present invention has an image processing control program recorded therein, the said program comprising the steps of inputting image data which represents information of each of the pixels resolved in a dot matrix form from an image, enlarging the luminance distribution in the image data, highlighting saturation in the image data, and suppressing the luminance distribution enlarging operation and the saturation highlighting operation with respect to each other.
In the present invention thus constructed, the contrast enlarging means enlarges the luminance distribution in the image data, while the saturation highlighting means highlights the saturation of each pixel. In this connection, the highlighting process suppressing means suppresses the luminance distribution enlarging operation and the saturation highlighting operation with respect to each other.
Thus, when contrast is enlarged by the contrast enlarging means and saturation is highlighted by the saturation highlighting means, the highlighting process suppressing means suppresses both highlighting operations with respect to each other to prevent a synergistic effect of affording a loud image. Moreover, even if after one adjustment there is made the other adjustment, it is possible to keep the previous adjustment effective.
The highlighting process suppressing means is not specially limited if only it can suppress both luminance distribution enlarging operation and saturation highlighting operation with respect to each other. In this connection, the luminance distribution enlarging operation can be done by various methods and this is also true of the saturation highlighting operation. Therefore, concrete methods may be selected suitably according to the contrast enlarging means and saturation highlighting means adopted. In suppressing both operations with respect to each other, it is not always necessary to suppress the two mutually. The suppressing method may be such that the suppressing process is applied from one to the other but is not applied from the other to one. By so doing, it becomes possible to make selection between the case where a synergistic highlighting is to be prevented and the case where it is allowed.
Needless to say, if the luminance distribution enlarging operation and the saturation highlighting operation can eventually be suppressed with respect to each other, this will do. For example, there may be adopted a method wherein when it is necessary to suppress the enlarging operation of the contrast enlarging means, the enlarging operation of the contrast enlarging means is not suppressed, but there may be performed a further transformation processing which denies the result of enlargement in the enlarged image data. It is also possible to obtain the same result by darkening the entire image though this is different from the enlargement of contrast. This can also be said of the saturation highlighting operation.
In connection with such a case, it is a further object of the present invention to provide a more concrete method for suppressing both highlighting operations with respect to each other.
In the image processing system according to the present invention, the highlighting process suppressing means sets a correlation so that when one of a parameter which represents the degree of luminance distribution enlargement in the contrast enlarging means and a parameter which represents the degree of saturation highlighting is large, the other is small.
In the present invention thus constructed, the contrast enlarging means transforms image data with use of the parameter which represents the degree of luminance distribution enlargement, and the saturation highlighting means also transforms image data with use of the parameter which represents the degree of saturation highlighting. Therefore, the correlation of one being large and the other small made by the highlighting process suppressing means eventually causes the luminance distribution enlarging operation and the saturation highlighting operation to suppress each other.
Thus, the correlation using the parameters representing the degree of contrast enlargement and the degree of saturation highlighting facilitates the processing in the present invention.
Various methods are available for correlating both parameters. For example, if one parameter is multiplied by a coefficient using the other parameter as denominator, the one parameter becomes smaller as the other parameter becomes larger. In this case, the other parameter is not influenced by the one parameter, but if it is desired that both be mutually influenced, this can be done easily by another method. Needless to say, there may be adopted a method wherein a transformation table to be referred to by both parameters and such transformation values as to suppress each other are read out.
Not only the above methods but also various other methods are available for the transformation of image data performed by the contrast enlarging means and the saturation highlighting means. To be concrete, the image data transforming operation may be done for each pixel. In the case where the image data transformation for contrast enlargement and the image data transformation for saturation highlighting are not performed for each pixel, a causal relation between both processing becomes complicated, and as the case may be it is required to conduct a complicated processing for suppressing the, two with respect to each other, or a work area may be needed separately. In contrast therewith, the image data transforming operation performed for each pixel is advantageous in that the influence of the image data increasing or decreasing process on contrast and saturation becomes simple and hence the mutually suppressing process also becomes simple.
Also in setting the degree of highlighting, one or both of the contrast enlarging means and the saturation highlighting means may analyze image data and set the degree of highlighting. That is, the contrast enlarging means and the saturation highlighting means set the degree of highlighting automatically and in the course of this automation the highlighting process suppressing means performs the foregoing suppressing operation. Therefore, while the contrast enlarging means sets the degree of highlighting, if the degree of highlighting is weakened or a change is made from one processing to another by reference to, for example, the parameter in the saturation highlighting means, such a processing itself leads to constitution of the highlighting process suppressing means. Needless to say, in the case where the degree of highlighting is weakened or a change is made from one processing to another by reference to, for example, the parameter in the contrast enlarging means, such a processing itself leads to constitution of the highlighting process suppressing means.
It goes without saying that not only for judging whether the image concerned is a natural picture or not on the basis of the number of colors but also for judging whether the image concerned is to be subjected to image processing or not, there may be adopted other methods.
For example, there may be adopted a method wherein a binary image data, such as monochrome image data, is determined on the basis of a luminance distribution and, if the image data concerned is a binary image data, the enlargement of luminance distribution and highlighting of saturation may be omitted.
As to a binary image, there is no luminance distribution in a substantial sense nor is saturation, so once a binary image data is determined from a luminance distribution, there is performed neither enlargement of luminance distribution nor highlighting of saturation. Further, since it is possible that a binary image data will have a certain color, there can be two luminances corresponding respectively to the presence and absence of that color. It is possible to judge whether a luminance is of that color or not, but when there is no suggestive information, it may be judged that the image data concerned is a binary black-and-white image data when the luminance distribution is concentrated on both ends in a reproducible range. That is, in the case of a black-and-white image, it can be said that the luminance distribution is concentrated on both ends in the reproducible range, thus permitting judgment.
It is the formation of a frame that can occur frequently in image processing. It is not necessary to apply any image processing to the frame. It is therefore a further object of the present invention to provide an image processing system, an image processing method, and a medium having an image processing control program recorded thereon, capable of performing an image processing most suitable to image data having a frame.
The image processing system according to the present invention inputs image data which represents information of each of pixels resolved in a dot matrix form from an image and then executes a predetermined image processing. The image processing system is provided with a frame discriminating means which, on the basis of the image data, judges a portion including an extremely large number of certain pixels to be a frame portion, and an image data excluding means which excludes from the image processing the image data of pixels having been judged to be a frame portion.
In the image processing method according to the present invention, a predetermined image processing is performed for image data which represents information of each of pixels resolved in a dot matrix form from an image. According to the same method, if a portion of the image data includes an extremely large number of certain pixels, it is judged to be a frame portion, and the image data of the pixels having been judged to be a frame portion is not subjected to the image processing.
Further, the medium according to the present invention has an image processing control program recorded thereon for inputting image data which represent information of each of the pixels resolved in a dot matrix form from an image and performing a predetermined image processing, which program comprises the steps of judging, on the basis of the image data, a portion to be a frame portion if an extremely large number of certain pixels are included therein, and excluding from the image processing the image data of the pixels having been judged to be a frame portion.
In the present invention thus constructed, the frame discriminating means judges, on the basis of the image data, a portion of the image data to be a frame portion if an extremely large number of pixels are included therein, and the image data excluding means excludes from the image processing the image data of the pixels having been judged to be a frame portion.
That an image possesses a frame occurs frequently. The pixels which constitute a frame are almost the same pixels and the number thereof is extremely large as compared with the number of the other pixels. Therefore, if they are included in an extremely large number in a certain portion of the image data, this portion is judged to be a frame portion and is not subjected to the image processing.
The frame which occurs in the image data is black or white in many cases. It is therefore a further object of the present invention to provide an image processing system capable of excluding such a black or white frame efficiently. In the image processing system according to the present invention, the frame discriminating means regards pixels which take both-end values in an effective range of the image data as a candidate for a frame portion.
Needless to say, the color of frame is not limited to black and white. It is a further object of the present invention to exclude even other frames than black and white frames.
In the image processing system according to the present invention, the frame determining means totals luminances of the pixels into a luminance distribution as a whole when the image data is a natural picture and regards a prominent luminance portion in the luminance distribution as a candidate for a frame portion.
If such a prominent luminance portion exists as a monocolor frame, only the luminance portion corresponding to that color projects. Therefore, a prominent luminance portion, if any, is judged to be a frame portion of image data, and once a frame portion is detected, the data of the frame portion need not be used in the enlargement of luminance distribution and highlighting of saturation. More particularly, if a prominent luminance portion is used as a criterion in the luminance enlargement, an effective judgment may no longer be feasible, so such a portion is judged to be a frame portion and is not used in the enlargement of luminance distribution.
As an example, the luminance portions concentrated on both ends in a reproducible range may be judged to be a white or black frame portion. A white or black frame is often detected and adopted and can also result from trimming. It corresponds to an end portion in a reproducible range. Therefore, the luminance portion concentrated on the end portion is judged to be a frame portion.
As an example of judging whether the image data is of a natural picture or not, there may be used a natural picture discriminating means which judges the image data to be not of a natural picture when the luminance distribution exists in the shape of a linear spectrum. It can be said that a natural picture is characterized by a smooth width of its luminance distribution. In most cases, therefore, if the luminance distribution is in the shape of a linear spectrum, one can judge that the image data is not of a natural picture.
In some case the image processing system is used alone, and in another case it may be incorporated in a certain apparatus. Thus, the idea of the invention is not restricted to only a limited case, but covers various modes. A change maybe made as necessary, for example, between software and hardware.
As an example, the image processing system in question may be applied to a printer driver in which inputted image data is transformed into image data corresponding to the printing ink used and the thus-transformed image data is printed using a predetermined color printer.
Thus, the printer driver transforms the inputted image data correspondingly to the printing ink used, and at this time the foregoing processing may be carried out for the image data. In the case where a materialized example of the idea of the invention takes the form of software in the image processing system, such a software is inevitably present and utilized also on the recording medium used in the system. It is therefore a further object of the present invention to provide not only the foregoing image processing system and method but also a medium having an image processing control program recorded thereon which program substantially controls the image processing system and method.
Needless to say, the recording medium maybe a magnetic recording medium or a magneto-optic recording medium, or any other recording medium which will be developed in the future. Just the same way of thinking is applicable also to duplicate products, including primary and secondary duplicate products. The present invention is applicable also to the case where a communication line is utilized as a software supply line. In this case, the software supply side using the communication line functions as a software supply system and thus the present invention is also utilized here.
Further, even a construction including software as a portion and hardware as another portion is by no means different from the concept of the present invention. There may be adopted a construction wherein image data are partially recorded on a recording medium and are read as necessary. It goes without saying that the invention is applicable also to such image processing systems as color facsimile machine, color copying machine, color scanner, digital still camera and digital video camera.